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1.0  EXECUTIVE  SUMMARY 


1.1  OBJECTIVES  OF  THE  DEMONSTRATION 

There  are  many  U.S.  Department  of  Defense  (DoD)  sites  where  unexploded  ordnance  (UXO) 
cleanup  is  difficult  because  current  technology  is  inadequate  in  forested  areas  and  in  challenging 
terrain.  The  man  portable  vector  (MPV)  is  a  handheld  technology  that  is  designed  to  extend  the 
classification  perfonnance  of  the  latest  vehicle-based  geophysical  platfonns  to  sites  where 
vegetation  or  terrain  limit  access  to  these  platforms.  The  objective  of  this  study  was  to 
demonstrate  through  data  collection  and  analysis  that  a  handheld  sensor  such  as  the  MPV  could 
be  utilized  for  detection  and  classification  of  shallow  UXO  at  live  sites.  The  MPV  was  first 
validated  at  the  Yuma  Proving  Ground  (YPG)  Standardized  Test  Site  and  subsequently 
participated  in  the  Environmental  Security  Technology  Certification  Program  (ESTCP) 
demonstration  of  man-portable  systems  at  the  fonner  Camp  Beale,  which  contained  areas  with 
forest  and  rolling  terrain.  The  performance  objectives  were  met  by  achieving  high  rates  of 
detection,  classification,  and  clutter  rejection. 

1.2  TECHNOLOGY  DESCRIPTION 

The  MPV  sensor  head  is  a  handheld,  wide-band,  time-domain,  electromagnetic  induction  (EMI) 
sensor  that  comprises  of  a  50-cm  diameter  transmitter  loop  and  an  array  of  five  three- 
dimensional  receivers.  This  second-generation  prototype  was  specially  designed  for  this  study  to 
improve  maneuverability  and  ruggedness  over  the  initial  sensor  that  was  developed  by  Cold 
Regions  Research  and  Engineering  Laboratory  (CRREL)  in  Strategic  Environmental  Research 
and  Development  Program  (SERDP)  Munitions  Response  (MR)  Project  MR- 1443  (O’Neill  and 
Barrowes).  The  MPV  is  field  programmable  through  a  user  interface  that  also  provides 
immediate  feedback  on  field  data  for  quality  control  (QC)  and  target  localization.  Generally 
operated  by  a  two-person  team,  the  MPV  can  be  operated  in  dynamic  survey  for  digital 
geophysical  mapping  (DGM)  and  target  detection,  and  for  cued  interrogation  of  detected 
anomalies.  In  cued  mode,  high  quality  data  are  collected  by  taking  multiple  static  soundings  for 
joint  interpretation  through  geophysical  inversion.  Reliable  inversion  requires  accurate 
positioning.  The  MPV  utilizes  a  dedicated,  local,  portable  positioning  receiver  station  that 
monitors  the  primary  transmitter  field  from  the  MPV — like  a  beacon — and  returns  relative 
location  estimates  with  cm-level  accuracy  out  to  a  range  of  4  m.  This  approach  overcomes  line- 
of-sight  constraints  of  standard  survey  methods  such  as  Global  Positioning  System  (GPS)  and 
roving  laser  rangers  and,  unlike  a  fixed  template,  facilitates  adaptive  choice  of  sounding 
locations  according  to  target  response  and  environmental  conditions. 

The  MPV  was  deployed  at  two  sites  by  Sky  Research,  Inc.  (SKY)  with  technical  assistance  from 
G&G  and  CRREL.  The  data  were  subsequently  analyzed  and  processed  by  SKY.  The  YPG 
demonstration,  which  occurred  in  October  2010,  was  aimed  at  verifying  detection  and 
classification  on  a  blind  grid  after  training  on  calibration  lanes.  The  survey  protocol  involved 
operating  in  a  detection  search  mode  until  a  target  was  encountered.  The  detection  survey  was 
then  interrupted  and  a  cued  survey,  designed  to  collect  sufficient  high  quality  data  for 
classification,  was  executed.  The  technology  was  further  tested  in  the  Desert  Extreme  area  for  its 
ease  of  use  and  ruggedness.  The  Camp  Beale  demonstration  was  conducted  in  June  2011  and 
tested  classification  at  a  live  site  through  cued  interrogation  of  912  anomalies  previously 
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identified  in  a  DGM  survey  using  an  EM-61-MKII  sensor.  The  Camp  Beale  targets  were  split 
between  a  forested  area,  where  vehicular  access  was  precluded,  and  an  open  field,  where 
performance  with  vehicular  platforms  could  be  compared. 

1.3  DEMONSTRATION  RESULTS 

High  detection  perfonnance  was  demonstrated  at  the  YPG  site  with  100%  and  90%  detection 
rates  for  UXO  within  0.3  m  and  1  m  depth,  respectively.  Successful  classification  perfonnance 
was  also  demonstrated  with  95  and  100%  of  detected  UXO  (within  0.3  and  1.0  m  depths 
respectively)  correctly  identified.  Beacon  position  estimates  were  confirmed  to  be  sufficiently 
accurate  to  meet  classification  requirements.  Sensor  hardware  proved  to  be  maneuverable  and 
generally  resilient.  Efficient  survey  protocols  were  established  and  an  average  daily  productivity 
rate  of  100-anomaly  characterizations  was  achieved.  The  system  was  deemed  fit  for  live-site 
testing. 

At  the  Camp  Beale  demonstration  site,  cued  interrogation  data  were  collected  within  the  planned 
schedule  with  a  daily  productivity  rate  of  90  anomalies.  To  address  potential  issues  with  highly 
magnetic  soils  at  the  site  and  to  mitigate  anomaly  location  uncertainties  additional  soundings 
were  collected,  thus  lowering  the  production  rate  below  that  achieved  at  YPG.  Data  were  pre- 
processed  and  made  available  to  external  data  analysts  for  advanced  processing  and 
classification.  Interpretations  conducted  by  SKY  and  several  other  research  groups  resulted  in 
the  identification  of  100%  of  the  targets  of  interest  while  avoiding  excavation  of  over  80%  of  the 
clutter.  All  demonstration  objectives  were  successfully  met. 

1.4  IMPLEMENTATION  ISSUES 

The  YPG  and  Beale  studies  constitute  the  first  step  at  establishing  the  perfonnance,  limitations, 
optimum  usage,  and  costs  of  the  MPV  technology.  The  initial  demonstrations  were  successful 
and  showed  a  strong  potential  for  detection  and  classification  at  shallow  depth  (up  to  1  m)  with  a 
handheld  technology.  Of  particular  note  is  the  significant  reduction  in  the  number  of 
nonhazardous  clutter  items  that  need  to  be  excavated:  over  80%  of  these  items  could  be  left  in 
the  ground.  Additional  surveys  are  scheduled  through  2012-2013  as  part  of  the  ESTCP  Live-Site 
Demonstration  program  (Projects  MR-201 158  and  MR-201228). 

Today  there  is  no  commercially  available  system  with  similar  capabilities,  though  there  is  some 
overlap  with  cart-based  and  man-portable  prototypes  that  are  currently  being  tested  with  the 
ESTCP  (such  as  the  Time  Domain  Electromagnetic  Towed  Array  Detection  System 
[TEMTADS]  2x2  and  the  handheld  Berkeley  UXO  Discriminator  [BUD]).  The  MPV  sensor 
itself  is  a  unique  prototype.  It  was  fabricated  by  G&G  with  U.S.  government  funding  using 
“standard”  components  similar  to  those  found  on  the  Geometries  MetalMapper  in  addition  to  a 
number  of  custom  features.  Maintenance  and  integration  of  surveying  sensors  such  as  GPS  and 
heading  sensors  currently  relies  on  the  involvement  of  G&G. 

Technology  transition  efforts  are  ongoing.  The  MPV  will  now  be  field-tested  without  the 
presence  of  the  technology  developer.  A  wider  range  of  surveying  sensors  (such  as  tilt  meters) 
will  be  made  compatible  with  the  MPV  so  that  legacy  sensors  can  be  replaced.  All  MPV  data 
that  have  and  will  be  collected  in  the  ESTCP  demonstration  will  continue  to  be  made  available 
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for  testing  of  inversion  and  classification  algorithms  and  for  training  of  analysts.  Commercial 
partners  are  scheduled  to  participate  and  be  trained  by  SKY  at  data  collection  and  interpretation 
in  upcoming  field  demonstrations.  Demonstration  reports  will  document  survey  protocols  and 
results  and  will  promote  adoption  of  the  MPV  technology  by  industry. 
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2.0  INTRODUCTION 


This  project  demonstrates  the  capability  to  perfonn  classification  of  buried  UXO  with  a  handheld 
sensor,  the  MPV  EMI  sensor.  The  MPV  technology  was  designed  to  extend  the  advanced 
classification  capabilities  of  new-generation  geophysical  platforms  to  sites  with  challenging 
surveying  conditions  and  thus  provide  a  solution  for  UXO  cleanup  at  most  land  locations.  This 
project  comprises  development  of  the  technology,  live-site  testing,  and  data  analysis  for 
classification. 

2.1  BACKGROUND 

Decades  of  training  have  left  millions  of  acres  of  UXO-contaminated  land  in  the  United  States. 
The  Defense  Science  Board  observed  in  2003  that  existing  methods  for  UXO  detection  generally 
caused  expensive  digging  of  abundant  scrap  and  suggested  that  use  of  classification  technology 
could  significant  reduce  remediation  cost.  ESTCP  received  funding  in  2006  to  stimulate  the 
“development  of  advanced,  sophisticated  discrimination  technologies  for  UXO  cleanup”  and 
initiated  a  discrimination  pilot  study  to  test  emerging  technology.  The  first  three  discrimination 
studies  showed  that  a  significant  fraction  of  nonhazardous  scrap  could  be  safely  left  in  the 
ground  when  combining  new-generation  sensors  and  advanced  classification  methods.  The  new 
technology  relied  on  the  deployment  of  large,  vehicular  and  cart-based  geophysical  platforms 
that  required  favorable  survey  conditions,  minimal  vegetation,  and  modest  topographical 
variations.  Many  DoD  sites  are  less  amenable  to  vehicular  deployment.  Terrain  and  vegetation 
conditions  (e.g.,  dense  forests  and  steep  terrain)  can  also  preclude  use  of  traditional  location 
systems  like  GPS  and  laser  ranging  by  disrupting  their  line-of-sight  requirements. 

The  MPV  was  designed  to  extend  UXO  classification  to  challenging  survey  environments.  The 
MPV  EMI  sensor  incorporates  new-generation  multi-axis  receivers,  electronics,  and 
programmability  into  a  handheld  fonn  factor  for  improved  portability  and  reduced  logistical 
requirements.  The  MPV  can  be  utilized  for  detection  and  classification.  Detection  can  be 
achieved  through  dynamic  data  collection  and  DGM.  Classification  relies  on  the  collection  and 
processing  of  high  quality  data,  which  are  generally  obtained  through  cued  interrogation,  where  a 
series  of  static  soundings  are  collected  near  a  detected  anomaly.  Classification  requires  accurate 
positioning.  In  environments  where  line-of-sight  requirements  cannot  be  met  and  preclude  use  of 
GPS  or  laser  ranging  methods,  the  MPV  is  accurately  located  with  a  MPV-dedicated  local 
positioning  system  that  was  designed  and  tested  for  this  project.  The  system  is  based  on 
monitoring  the  MPV  EMI  field — like  a  beacon — from  a  portable  base  station  with  receivers. 
This  project  tested  the  second-generation  MPV.  The  first  MPV  prototype  was  developed  under 
the  SERDP  Project  MR- 1443  originally  led  by  Dr.  Kevin  O’Neill  and  Dr.  Benjamin  Barrowes. 
The  MPV  showed  a  strong  potential  for  UXO  classification  in  preliminary  field  tests  and  was 
redesigned  for  improved  ruggedness  and  maneuverability  as  part  of  the  original  SERDP  project 
and  this  project. 

The  demonstration  design  consisted  of  testing  the  MPV  at  a  controlled  site  in  the  first  year  and  at 
a  live  site  in  the  second  year.  The  first  demonstrated  was  set  at  the  UXO  Standardized  Test  Site 
at  YPG,  AZ,  in  October  2010.  The  MPV  was  tested  for  the  capabilities  to  endure  an  extended 
field  survey  and  achieve  satisfactory  detection  and  classification  performance  in  a  systematic, 
controlled  experiment.  The  second  study  tested  classification  at  a  live  site  in  June  2011  as  part  of 
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the  ESTCP  demonstration  at  former  Camp  Beale,  CA,  where  the  perfonnance  of  emerging 
portable  technologies  was  tested  in  areas  with  terrain  features  and  variable  forest  density. 

2.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  goal  of  this  demonstration  was  to  implement  the  MPV  technology,  to  test  the  capability  to 
carry  out  effective  surveys  at  live-munitions  sites  and  to  assess  data  quality  for  the  detection  and 
classification  of  buried  munitions.  The  technology  was  tested  by  field  deployment  and  data 
analysis  for  two  sites  with  specific  purposes. 

2.2.1  First  Demonstration  at  Standardized  Test  Site 

The  first  demonstration  took  place  at  YPG  and  comprised  of  two  main  objectives:  (1)  to  achieve 
high  performance  at  detection  and  classification  and  (2)  to  test  field  survey  procedures  at  an 
environmentally  easy  site.  Detection  and  classification  were  practiced  on  the  Calibration  Lanes, 
where  survey  parameters  and  protocols  were  verified  and  a  classifier  was  trained,  and  applied  to 
the  Blind  Grid  where  400  potential  anomalies  had  to  be  classified. 

The  main  objectives  were  largely  met  and  the  demonstration  was  a  success.  Detection  objective 
of  80%  was  exceeded  with  100%  target  detection  within  0.3  m  and  90%  within  1  m  depth  range 
and  no  false  alanns.  Classification  goals  of  80%  were  met  with  over  90%  correct  classification  of 
UXO  within  1  m  depth.  The  false  alarm  rate  for  classification  was  higher  (60-75%)  than  the 
stated  objective  (50%)  because  the  UXO-to-clutter  ratio  was  orders  of  magnitude  greater  than 
that  of  a  live  site.  Beacon  positioning  was  tested  relative  to  real-time  kinematic  (RTK)  GPS  and 
yielded  similar  cm-level  accuracy,  which  met  accepted  requirements  for  classification.  Field- 
readiness  was  also  tested.  The  technology  was  generally  straightforward  to  operate.  The 
hardware  was  resilient  to  the  2  week  field  deployment,  with  only  some  minor  damage  to  a  few 
connectors.  In  terms  of  productivity,  a  daily  average  of  100  anomalies  was  achieved  on  the  Blind 
Grid.  Data  were  processed  and  interpreted  within  the  expected  4  weeks  of  survey  completion. 

2.2.2  Live-Site  Demonstration 

The  second  demonstration  was  set  at  former  Camp  Beale,  a  live  site  that  was  chosen  for  the 
specific  purpose  of  testing  classification  with  man-portable  systems  in  areas  with  sloped, 
forested  terrain  where  vehicular  platforms  could  not  survey  (Figure  1).  Cued  interrogation  data 
were  collected  over  912  anomalies  while  using  the  beacon  system  for  positioning.  The  data  were 
preprocessed  and  made  available  to  other  research  groups  for  analysis  and  classification. 
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Figure  1.  Survey  with  MPV  and  beacon  positioning  boom  at  former  Camp  Beale. 


The  demonstration  was  successful.  High  quality  EMI  data  were  collected  with  accurate 
positioning  and  allowed  all  analysts  to  identify  all  buried  UXO  and  leave  over  80%  of  the  clutter 
in  the  ground.  The  data  collection  was  completed  well  within  the  planned  schedule.  The  MPV 
and  beacon  endured  the  field  survey  without  any  major  breakdowns — some  cable  connectors 
attached  to  the  control  unit  got  strained  and  were  replaced.  The  initial  productivity  objective  was 
100  targets  per  day.  Though  100  targets  were  collected  on  one  day,  the  daily  average  was  90 
targets.  The  objective  can  be  considered  to  be  met  given  that  the  survey  protocol  had  to  be 
amended  to  increase  the  number  of  soundings  per  anomaly  to  accommodate  offsets  between 
flagged  and  detected  target  locations  and  sample  the  significant  and  variable  background 
response  from  magnetic  soil. 

2.3  REGULATORY  DRIVERS 


The  Defense  Science  Board  Task  Force  on  UXO  noted  in  its  Fiscal  Year  2003  (FY03)  report  that 
75%  of  the  total  cost  of  a  current  clearance  is  spent  on  digging  scrap.  A  reduction  in  the  number 
of  scrap  items  dug  per  UXO  item  from  100  to  10  could  reduce  total  clearance  costs  by  as  much 
as  two-thirds.  Thus,  classification  efforts  focus  on  technologies  that  can  reliably  differentiate 
UXO  from  items  that  can  be  safely  left  undisturbed. 

Classification  only  becomes  a  realistic  option  when  the  cost  of  identifying  items  that  may  be  left 
in  the  ground  is  less  than  the  cost  of  directly  digging  them.  Because  classification  generally 
requires  a  detection  survey  as  a  precursor  step,  the  investment  in  additional  data  collection  and 
analysis  must  result  in  sufficient  clutter  rejection  to  recuperate  the  investment.  Even  with  perfect 
detection  performance  and  high  signal-to-noise  ratio  (SNR)  values,  successfully  sorting  the 
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detections  into  UXO  and  nonhazardous  items  is  a  difficult  problem  but,  because  of  its  potential 
payoff,  one  that  is  the  focus  of  significant  current  research.  This  demonstration  represents  an 
effort  to  transition  a  promising  classification  technology  into  widespread  use  at  UXO- 
contaminated  sites  across  the  country. 
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3.0  TECHNOLOGY 


The  MPV  technology  is  based  on  EMI  sensing  using  one  transmitter  coil  and  multiple  vector 
receivers  in  a  handheld  form  factor.  The  sensor  presented  in  this  study  is  the  second-generation 
prototype  MPV. 

3.1  MPV  TECHNOLOGY  DESCRIPTION 

3.1.1  Electromagnetic  Sensor 

The  MPV  is  a  handheld  sensor  with  wide-band,  time-domain,  EMI  technology.  The  sensor  head 
is  composed  of  a  single  transmitter  coil  and  an  array  of  five  receiver  units  that  measure  all  three 
components  of  the  electromagnetic  field  (Figure  2).  This  second-generation  MPV  was 
specifically  designed  to  (1)  be  man  portable  and  therefore  easy  to  deploy,  maneuver,  and  adapt  to 
a  survey  environment,  and  (2)  acquire  data  that  is  suitable  for  discriminating  UXO  from  non- 
UXO  targets.  The  MPV  head  is  a  50  cm  diameter  transparent  disk.  The  transmitter  coil  is  wound 
around  the  disk  and  intermittently  illuminates  the  subsurface.  Five  multi-component  receiver 
units  (cubes)  measure  the  three  orthogonal  components  of  the  transient  secondary 
electromagnetic  field  decay  with  three  air-induction  8  cm  square  coils.  One  receiver  cube  is  co¬ 
axial  with  the  transmitters  while  four  receivers  are  placed  off  axis  around  the  transmitter  loops  in 
a  cross  pattern.  Having  multiple  receivers  generally  improves  the  recovery  of  target  parameters 
for  classification  (Gasperikova  et  ah,  2007). 


Figure  2.  The  second  generation  MPV  with  beacon  positioning. 

Positioning  can  be  achieved  with  GPS  (for  open  field  survey)  or  beacon  receivers  (for  survey  in  a  forest 
or  in  steep  terrain).  Left  inset  shows  data  acquisition  system  (DAQ)  and  power  unit  mounted  on  a 
backpack  frame.  Right  panel  shows  view  of  sensor  head  from  above  with  cube  numbers. 

The  MPV  is  a  programmable  instrument.  The  duration  of  the  excitation  and  time  decay  recording 
can  be  adjusted  to  any  given  time  to  accommodate  survey  needs.  The  MPV  is  generally  with  two 
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main  modes:  detection  and  classification.  Detection  survey  consists  of  dynamic  data  collection 
for  DGM  using  fast  EMI  transmit-receive  cycles  so  that  the  sensor  can  continuously  move  (e.g., 
1  millisecond  [ms]  time  decay,  similar  to  Geonics  EM-61).  Discrimination  maximizes  data 
quality  for  classification.  The  sensor  is  static  to  stack  the  recorded  signal  and  reduce  noise. 
Longer  EMI  cycles  are  applied  to  capture  variations  in  time  decay  rates  (e.g.,  25  ms,  similar  to 
Geonics  EM-63).  This  late-time  information  has  been  shown  to  improve  distinction  between 
intact  ordnance  and  thinner  walled  shrapnel  and  cultural  debris  (Billings  et  ah,  2007). 

The  MPV  is  a  handheld  sensor.  The  sensor  head  weighs  12  pounds  and  the  backpack-mounted 
DAQ  and  batteries  weigh  35-40  pounds.  Existing  sensors  with  multiple  time  channel 
measurement  capabilities  (e.g.,  BUD,  Geonics  EM63,  TEMTADS)  are  required  to  be  mounted 
on  a  cart  platform  due  to  the  size  and  weight  of  the  multiple  coils  of  wire  required  for  the 
transmitters  and  receivers. 

The  MPV  user  interface  has  real-time  data  monitoring  capabilities.  The  recorded  data  can  be 
displayed  to  verify  data  quality  and  detect  potential  disturbances  such  as  presence  of  magnetic 
soil  or  a  damaged  receiver.  The  past  and  present  sensor  location  is  displayed  on  a  map  along  with 
preset  survey  points  to  verify  spatial  coverage  and  global  location.  A  target  detection  and 
location  tool  indicates  the  origin  of  measured  EMI  fields  with  arrows  (the  so-called  “dancing 
arrows”  in  top  left  comer  of  Figure  3).  These  features  assist  the  field  operator  in  efficient  data 
collection,  so  that  detection  and  discrimination  data  can  be  collected  as  part  of  the  same  survey, 
thus  limiting  the  need  to  revisit  an  anomaly  for  further  characterization. 


Figure  3.  The  MPV  detection  display  window  in  dynamic  data  collection  mode. 

The  top  left  panel  indicates  with  arrows  the  direction  of  the  nearest  compact  metallic  object  relative  to  the 
MPV  receiver  cubes  and  directs  the  operator  to  the  target  (here  the  MPV  sits  atop  the  target).  The  top 
middle  panel  shows  a  field  map  with  the  MPV  location  (red  dot)  and  azimuth  (black  line),  potential  target 
locations  (blue  dots),  and  cued  interrogation  soundings  (green  crosses).  The  middle  left  panel  provides 
feedback  on  positioning  sensors.  The  bottom  panel  has  the  data  acquisition  options  for  simple  one-button- 

touch  operation. 
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3.1.2  Geolocation 


The  sensor  requires  positioning  for  detection  and  classification,  though  with  different  spatial 
accuracy  requirements  for  each.  Therefore  a  field  survey  with  the  MPV  can  utilize  two 
complementary  positioning  systems.  Detection  mapping  has  decimeter  accuracy  requirements 
and  can  be  performed  with  a  GPS  or  a  spool-mounted  cotton  thread  and  optical  encoder. 
Classification  is  based  on  geophysical  inversion  of  multiple  soundings  and  generally  requires 
cm-level  sensor  positioning  when  surveying  a  target  (Bell,  2005).  Being  designed  to  extend 
UXO  classification  to  difficult  survey  environments,  the  MPV  technology  was  augmented  with  a 
local  positioning  system  that  remains  accurate  in  steep  terrain  and  under  thick  tree  canopy,  where 
GPS  cannot  guarantee  high  accuracy.  We  implemented  the  MPV-beacon  positioning  system  for 
cued  target  interrogation  (San  Filippo  et  ah,  2007;  Lhomme  et  ah,  2011).  The  operating  principle 
consists  of  locating  the  origin  of  the  MPV  transmitter,  acting  as  a  beacon,  with  a  pair  of  EMI 
receivers  rigidly  attached  to  a  portable  beam  that  serves  as  a  base  station  (Figure  2).  The  azimuth 
of  the  MPV  and  boom  are  recorded  with  a  3-component  attitude  sensor.  Field  trials  showed  1-2 
cm  and  1 -2 E  accuracy  for  position  and  orientation — similar  to  GPS — out  to  distances  of  3-4  m. 

3.2  MPV  TECHNOLOGY  DEVELOPMENT 

The  project  was  initiated  in  2005  under  SERDP  MR- 1443  and  led  by  Drs.  Kevin  O’Neill  and 
Benjamin  Barrowes  of  the  CRREL  Engineering  Research  and  Development  Center  (ERDC)  in 
Hanover,  NH.  The  first  MPV  prototype  was  built  in  2005-2006  by  David  George  of  G&G 
Sciences,  Grand  Junction,  CO.  It  was  tested  in  2007  at  ERDC  in  a  laboratory  setting.  Data 
analysis  showed  that  stable  target  parameters  could  be  retrieved  and  used  for  UXO  classification. 

The  SERDP  project  was  first  extended  in  2008  to  continue  testing.  The  first  field  tests  took  place 
at  the  SKY  test  plot  in  Ashland,  OR,  in  the  summer.  A  series  of  UXO  was  surveyed  in  static  and 
dynamic  mode  using  a  template  with  marked  locations  and  the  ArcSecond  laser  ranging  system. 
Data  analysis  confirmed  stable  target  parameter  recovery  for  classification.  The  effect  of 
magnetic  soil  on  EMI  sensors  was  investigated.  Adverse  soil  effects  could  be  defeated  owing  to 
the  MPV’s  array  structure.  The  ArcSecond  laser  positioning  technology  that  had  supported  early 
MPV  tests  proved  to  be  impractical  and  not  reliable  enough  for  field  application  due  to  the 
requirement  to  keep  line-of-sight  with  all  three  rovers,  and  long  setup  and  calibration  time.  This 
experience  stimulated  development  of  an  alternative  positioning  method  based  on  the  beacon 
principle. 

The  SERDP  project  was  extended  by  SKY  in  2009  to  test  the  beacon  concept  and  prepare 
modification  of  the  original  MPV  prototype  for  extensive  field  deployments.  The  sensor  head 
was  redesigned  with  lighter  materials  and  a  smaller  head  diameter  to  reduce  weight  and  improve 
maneuverability  while  maintaining  its  expected  perfonnance  (Appendix  D  of  Lhomme,  2011). 
The  receiver  cubes  were  brought  inside  the  transmitter  coil  to  reduce  fragility,  and  transparent 
material  was  employed  to  allow  the  operator  to  see  the  ground  through  the  unit.  Actual 
fabrication  of  the  new  head  and  replacement  of  the  DAQ  was  funded  through  the  SERDP  project 
and  this  ESTCP  project.  The  sensor  head  was  lightened  from  23  to  12  pounds. 
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3.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  MPV  TECHNOLOGY 

The  MPV  is  the  only  available  non-cart-based  system  that  can  acquire  multistatic, 
multicomponent  data  on  a  wide  and  programmable  range  of  time  channels.  The  MPV  offers 
several  key  benefits: 

•  Handheld  fonn  factor:  The  MPV  can  be  deployed  at  sites  where  terrain  and 
vegetation  preclude  use  of  heavier,  cart-based  systems.  Portability  can  improve 
productivity  in  rough  terrain.  The  system  is  easy  to  pack  and  can  ship  with 
standard  regular  courier  services; 

•  Five  receivers  simultaneously  record  three  orthogonal  components  of 
electromagnetic  field  with  near-perfect  relative  positioning  among  receivers. 
Multicomponent,  multi-axis  design  reduces  number  of  soundings  for  target 
characterization  and  relaxes  positional  accuracy  (Grzegorczyk  et  ah,  2009).  Test 
with  low-noise  test-stand  MPV  (first  generation)  data  showed  that  UXO  could  be 
identified  with  as  few  as  5  soundings  (Barrowes  et  ah,  2007).  This  was  confirmed 
at  YPG  and  Camp  Beale. 

•  Magnetic  soil  can  be  detected  and  mitigated:  The  geometric  arrangement  of 
receivers  and  the  wide-band  time  range  offer  potential  for  identifying  and 
neutralizing  the  effect  of  magnetic  soil  through  techniques  developed  in  SERDP 
MR-1414  and  MR-1573. 

•  Fully  programmable  through  field  display:  Graphical  field-user  interface  controls 
acquisition  parameters  such  as  transmitter  wavefonn  characteristics,  duration  of 
excitation,  number  of  measurement  cycles,  stacking,  and  recorded  time  channels. 

•  Highly  stable  EMI  components:  Responses  are  directly  predictable  using  standard 
EMI  theory.  Field  tests  verified  that  MPV  components  had  imperceptible 
measurement  drift  and  were  largely  insensitive  to  survey  conditions. 

•  Small  target  characterization:  Small  items  have  localized,  rapidly-varying  spatial 
response.  Voltage  in  an  air  induction  receiver  coil  is  an  average  of  a  target 
scattered  field  through  the  face  of  the  loop.  Therefore,  large  receivers  tend  to 
“smear  out”  secondary  fields.  The  MPV’s  8  cm  square  coils  are  typically  smaller 
than  most  multichannel  sensors  (e.g.,  Geonics  EM-63  has  50  cm  loops  and 
TEMTADS  has  25  cm)  and  thus  better  suited  to  detecting  and  sampling  signals 
from  small  targets. 

The  MPV  has  some  limitations.  Because  there  is  only  one  transmitter,  multiple  soundings  must 
be  collected  to  characterize  a  target.  Therefore  the  MPV  requires  (1)  an  accurate  positioning 
system  for  cued  interrogation  and  (2)  manual  intervention  to  move  the  sensor,  which  reduces 
productivity  relative  to  a  multitransmitter  platform  for  which  a  single  sounding  is  often 
sufficient. 
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4.0  PERFORMANCE  OBJECTIVES 


The  MPV  studies  presented  herein  cover  data  collection  and  data  analysis  for  classification.  The 
perfonnance  objectives  were  similar  for  the  YPG  Blind  Grid  and  the  Camp  Beale  site.  The 
classification  problem  was  relatively  more  difficult  at  the  YPG  test  site  than  at  a  live  site  because 
of  the  low  clutter  rate  and  the  large  number  of  target  types.  Results  are  presented  in  Table  1. 
Success  is  indicated  in  the  “Results”  column  with  a  bold  font  to  help  readability. 

Table  1.  Performance  objectives. 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Results 

Quantitative  Performance  Objectives 

Detection  of  all 
munitions  of 
interest 

Percent  detected  of 
seeded  items 

•  Location  of  seeded 
items 

•  Prioritized  dig  list 

YPG  only:  Probability  of 
detection  (Pd)>0.90 

Pd=l  within  0.3  m  depth 
and  Pd=0.90  within  1  m 

Correct 

classification  of 
munitions 

Rate  of  identified 
munitions 

•  Prioritized  dig  list 

Probability  of  correct 

UXO  classification  (Pc): 
YPG:  Pc>0.90 

Beale:  Pc>0.95 

YPG:  Pc>0.90 

Beale:  Pc=l 

Reduction  of  false 
alarms 

Number  of  false 
targets  eliminated  at  a 
specified  confidence 
level 

•  Prioritized  dig  list  with 
probabilities 

•  Validation  data  for  all 
selected  targets 

Reduction  of  false  alarms 
by  more  than: 

•  50%  at  YPG 

•  40%  at  Beale 

YPG:  40%  and  25%  (0.3  and 

1  m) 

Beale:  over  80% 

Location  accuracy 

Average  error  and 
standard  deviation  in 
northing,  easting,  and 
depth  for  validated 
anomalies 

•  Validated  targets 
surveyed  with  1  cm 
accuracy 

•  Estimated  location 
from  data  analysis 

YPG:  AN,  AE  and  AZ 
<0.25  m 

Beale:  AN,  AE  <0.05  m 
and  AZ  <0.10  m 

YPG:  AN,  AE  and  AZ  <0.05 

m 

Beale:  AN,  AE  <0.10  m  and 
AZ  <0.05  m 

Production  rate 

Acreage  of  data 
collection  per  day 

Time  required  to 
analyze  each  target 

•  Log  of  field  work  and 
data  analysis  time 
accurate  to  1 5  minutes 

Survey:  100  anomalies  per 
day 

Preprocessing  time:  3 
minutes  per  target 

YPG:  100  targets/day 

Beale:  90  targets/day 

Beale:  Data  preparation  of  5 
minutes/target 

Qualitative  Performance  Objectives 

Field  usability 

•  Feedback  from  field 
technician 

•  Training  time 

•  Rate  of  anomalies 
requiring  resurvey 

•  Easy  operation 

•  Difficult  trouble¬ 
shooting/maintenance 

•  Quick  training 

•  Resurvey:  0  at  YPG,  1% 
at  Beale 

4.1  OBJECTIVE:  DETECTION  OF  ALL  MUNITIONS  OF  INTEREST 

Detection  was  only  tested  at  YPG.  The  objective  for  a  handheld  sensor  was  to  detect  shallow 
targets.  The  objective  was  met  with  100%  detection  of  munitions  within  0.3  m  depth  and  90% 
within  1  m  depth.  Note  that  some  targets  were  also  detected  beyond  1  m  depth.  There  was  no 
false  detection  from  any  nonmetallic  object. 

4.2  OBJECTIVE:  CORRECT  CLASSIFICATION  OF  MUNITIONS 

All  shallow  munitions  should  be  recommended  for  excavation.  This  objective  relied  both  on  data 
collection  and  analysis.  It  required  that  sufficient,  high-quality  data  would  be  collected,  and  that 
the  ensuing  data  analysis,  inversion,  and  classification  process  identified  munitions.  The 
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objective  was  met  at  both  sites:  95%  of  the  seeded  munitions  were  identified  within  0.3  m  depth 
and  90%  between  0.3-1  m  at  YPG  (100%  of  detected  targets);  Camp  Beale  had  100%  success. 

4.3  OBJECTIVE:  REDUCTION  OF  FALSE  ALARMS 

The  ultimate  goal  of  classification  is  to  identify  all  potential  munitions  and  leave  a  maximum  of 
clutter.  Our  initial  goal  was  a  clutter  rejection  rate  of  40-50%.  At  YPG  that  objective  was  not 
realistic  as  the  clutter-to-munitions  ratio  was  orders  of  magnitude  larger  than  at  a  live  site.  The 
reported  clutter  rejection  rates  were  40%  at  0.3  m  depth  and  25%  beyond,  which  is  acceptable — 
difficulty  of  classification  is  compounded  by  the  large  number  and  variety  of  munitions  (14 
potential  types  instead  of  three  to  five,  typically).  The  results  at  Camp  Beale  are  promising  with 
over  80%  clutter  rejection  at  stop-dig  point. 

4.4  OBJECTIVE:  LOCATION  AND  DEPTH  ACCURACY 

Classification  relies  on  reliable  target-parameter  extraction,  including  target  location  and  depth. 
Accurate  prediction  of  target  location  and  depth  is  also  important  for  safe  and  efficient 
remediation.  The  objective  was  met  at  both  sites  as  target  location  and  depth  were  predicted 
within  0.1  m,  which  is  less  than  half  the  width  of  a  typical  shovel  blade  for  all  munitions. 

4.5  OBJECTIVE:  PRODUCTION  RATE 

Cued  interrogation  survey  and  data  analysis  should  be  quicker  than  excavating  every  potential 
target.  Here  the  productivity  objective  focused  on  the  specific  tasks  of  data  collection  and  data 
preprocessing  (before  geophysical  inversion).  The  latter  task  included  digitizing  field  notes, 
consolidating  all  soundings  for  each  anomaly,  inferring  sensor  position  with  GPS  or  beacon, 
merging  with  electromagnetic  data  and  correcting  for  background  and  transmitter  current 
intensity. 

The  objective  was  to  characterize  100  anomalies  per  day  with  a  mean  preprocessing  time  of  3 
minutes  per  target.  The  averages  were  100  anomalies  per  day  at  YPG  and  90  at  Camp  Beale, 
where  the  number  of  soundings  per  anomaly  was  increased  from  seven  to  12  to  ensure  sufficient 
spatial  coverage  and  frequently  sample  the  geologic  background.  Preprocessing  time  was 
generally  slower  because  the  process  was  still  experimental.  At  Beale  the  treatment  of  soil 
response  added  post-survey  QC  tasks  to  validate  background  soundings  against  a  soil  model; 
beacon  verification  procedures  also  had  to  be  revised  to  accommodate  complex  terrain. 

4.6  OBJECTIVE:  FIELD  USABILITY 

Field  usability  consisted  of  qualitative  assessment  of  the  technology  by  field  users.  This 
objective  is  important  for  industry  adoption.  It  was  found  that  it  was  generally  easy  to  set  up  and 
operate  the  technology;  ergonomics  and  maneuverability  were  adequate  though  the  DAQ/power 
backpack  felt  too  heavy;  hardware  seemed  resilient  to  field  use  though  some  connectors  were 
weak;  troubleshooting  and  field  maintenance  were  difficult  and  vastly  relied  on  the  sensor 
manufacturer.  Details  are  presented  in  Section  8.5. 
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5.0  SITE  DESCRIPTION 


This  section  provides  a  brief  summary  of  the  conditions  in  which  the  demonstrations  took  place 
at  YPG  and  fonner  Camp  Beale.  Further  infonnation  can  be  found  on  the  U.S.  Army 
Environmental  Command  website  (http://aec.army.mil/usaec/technology/uxo01.html)  for  YPG 
and  the  Site  Inspection  Report  for  Beale. 

5.1  SITE  LOCATION  AND  HISTORY 

The  first  site  was  chosen  as  a  standardized  validation  site  for  DoD  and  the  second  by  ESTCP  in 
the  context  of  the  Live-Site  Demonstration  program  as  the  next  in  a  progression  of  increasingly 
more  complex  sites  for  demonstration  of  the  classification  process.  The  YPG  Standardized  Test 
Site  is  located  near  Yuma,  AZ.  The  site  was  chosen  over  Aberdeen  Proving  Ground  mostly  for 
weather  considerations  to  maximize  the  chance  of  favorable  conditions  in  October.  Three  areas 
were  surveyed  at  YPG:  the  calibration  lanes  are  set  for  training  and  contain  a  wide  selection  of 
targets  whose  type  and  location  are  known  to  the  demonstrators.  The  blind  grid  is  used  for 
testing.  The  desert  extreme  area  is  covered  with  desert-type  vegetation  and  used  to  test  the 
perfonnance  of  different  sensor  platforms  in  a  more  severe  desert  environment.  The  calibration 
lanes  and  blind  test  grids  measure  30  m  by  40  m  and  40  m  by  40  m,  respectively.  The  desert 
extreme  area  is  located  southeast  of  the  open  field  site  and  has  dimensions  of  50  m  by  100  m. 

The  Camp  Beale  site  is  located  near  Marysville,  CA,  and  was  selected  for  demonstration  because 
it  was  composed  of  hilly  open  field  and  forests  and  it  was  thought  to  contain  a  wide  mixture  of 
munitions.  These  features  increased  site  complexity  and  were  likely  to  be  encountered  on 
production  sites.  The  tree  cover  posed  a  navigation  challenge  by  increasing  the  difficulty  of 
obtaining  accurate  GPS  readings.  The  broader  fonner  Camp  Beale  area  currently  consists  of 
multiple  land  use  property  areas.  It  includes  cattle  grazing  and  ranching,  some  rural  residential 
use,  and  the  central  Spenceville  Wildlife  and  Recreation  Area.  In  terms  of  military  use,  Camp 
Beale  was  selected  in  1943  as  the  site  for  the  West  Coast  Chemical  Warfare  School  (CWS),  and 
then  became  in  1945  a  point  for  classification,  rehabilitation,  and  repacking  of  the  CWS  materiel. 
In  1947,  the  War  Assets  Administration  assumed  custody  with  certain  regions  being  reserved  for 
use  by  the  National  Guard.  In  1948  the  Air  Force  acquired  the  land  (designated  it  as  Beale  Air 
Force  Base)  to  train  bombardier-navigators  in  radar  techniques.  By  1956,  the  Navy  began  using 
two  of  the  target  areas  until  1957,  when  there  was  no  more  need  for  the  bombing  areas  and 
ranges.  At  that  time,  a  large  portion  of  the  site  (approximately  65,000  acres)  was  declared  excess. 
The  demonstration  area  spread  over  10  acres  divided  into  sub-areas  for  the  MetalMapper,  the 
portable  systems,  and  a  combined  area  where  both  collected  data.  An  EM-61  cart  surveyed  all 
three  sub-areas  at  100%  coverage.  The  50-acre  demonstration  site  is  shown  in  Figure  4  with  the 
extent  of  the  three  sub-areas  shown.  Shape  files  delineating  the  final  sub-area  boundaries  are 
available  from  the  ESTCP  Office. 
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Figure  4.  Camp  Beale  site  overview  with  delineation  of  survey  grids  for  each  sensor  type. 

5.2  SITE  GEOLOGY 

The  YPG  site  is  located  in  the  desert  and  consists  of  sand  and  pebbles  with  minimal  effect  on  an 
EMI  sensor.  Minimal  infonnation  was  available  for  Camp  Beale,  though  red  soils  were  noticed 
during  the  site  visit.  Red  soils  are  often  magnetite  rich  and  can  therefore  affect  EMI  sensors. 

5.3  MUNITIONS  CONTAMINATION 

The  YPG  site  contains  a  wide  selection  of  standard  projectiles,  mortars,  rockets  and  sub¬ 
munitions:  155  mm,  two  types  of  105  mm,  81  mm,  2.75-inch,  60  mm,  57  mm,  37  mm,  40  mm, 
20  mm,  M75,  BDU28,  M42,  and  BLU26.  The  calibration  portion  of  the  test  site  consists  of  19 
lanes,  most  of  which  containing  six  identical  munitions  buried  in  various  orientations  and  at 
three  different  depths.  There  are  also  steel  spheres  buried  at  a  depth  of  0.5  to  2  m,  circular  steel 
plates  buried  at  0.3  m  and  0.9  m,  copper  wire  hoops  of  various  diameter  and  gauge  at  0.3  m 
depth.  The  first  and  last  opportunity  of  each  Calibration  Lane  contains  a  3.6  kg  steel  ball  buried 
at  1 5  cm  to  provide  a  unifonn  signature  that  can  be  identified  when  looking  at  raw  data.  Any  of 
these  items  could  be  encountered  on  the  Blind  Grid. 

At  Camp  Beale,  the  particular  demonstration  area  showed  evidence  of  81  mm  mortars  and  105 
mm  projectiles  that  were  found  during  the  Site  Inspection  intrusive  investigation  in  2005.  It  was 
also  suspected  that  60  mm  mortars  may  be  present,  as  well  as  37  mm  projectiles  that  had  been 
found  scattered  throughout  the  former  Camp  Beale.  Due  to  the  complex  historical  usage  of  this 
site  over  many  years  and  the  overlapping  network  of  historical  ranges  throughout,  it  was  also 
likely  that  other  munitions  types  beyond  those  listed  above  may  be  encountered.  The  principal 
list  of  targets  of  interest  (TOI)  comprised  of  105  mm,  81  mm,  60  mm,  37  mm,  and  industry 
standard  object  (ISO)  (1. 25-inch  steel  cylinders). 
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6.0  TEST  DESIGN 


The  goal  of  the  study  was  to  demonstrate  and  characterize  detection  and  discrimination  with  the 
MPV.  Sensor  discrimination  performance  was  characterized  as  a  function  of  the  size  and  depth 
of  the  buried  targets  and  the  presence  and  effect  of  aggravating  factors  (nearby  object,  magnetic 
soil,  and  complex  terrain).  This  section  describes  data  collection  and  analysis. 

6.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

At  YPG,  the  Calibration  Lanes  were  used  to  define  survey  protocols  for  detection  and  cued 
interrogation.  The  survey  method  consisted  of  a  one-pass  survey  in  which  a  detection  search  is 
performed  until  an  anomaly  is  detected  and  approximately  localized,  at  which  point  the  search  is 
momentarily  interrupted  for  cued  interrogation,  where  a  series  of  five  to  10  static  soundings  are 
acquired  on  top  of  and  around  an  anomaly.  Key  parameters  such  as  time  ranges,  detection 
threshold,  and  number  of  soundings  for  cued  survey  were  tested.  Given  that  the  target  type, 
location,  and  depth  were  known,  sensor  perfonnance  could  be  immediately  evaluated.  The  data 
were  analyzed,  inversion  parameters  were  tested,  classification  strategies  were  assessed,  and 
classifiers  were  trained.  The  calibrated  procedures  were  subsequently  applied  to  the  Blind  Grid, 
where  detection  and  classification  were  to  be  evaluated  and  scored. 

The  Camp  Beale  study  focused  on  classification  through  cued  interrogation.  Training  was 
initially  performed  at  a  test  pit,  where  data  were  collected  over  site -representative  targets — 37, 
60,  and  81  mm  mortars,  105  mm  projectiles,  and  a  4-inch-long  steel  cylinder  also  known  as 
ISO — and  scrap  buried  in  at  least  three  different  orientations.  Calibration  was  also  perfonned  on 
a  daily  basis  by  surveying  over  an  Instrument  Verification  Strip  (IVS),  where  one  instance  of  the 
5  target  types  was  buried.  The  IVS  was  emplaced  to  verify  data  quality,  stability  of  the  target 
responses  and  accuracy  of  beacon  positioning. 

6.2  SITE  PREPARATION 

The  U.S.  Anny  Aberdeen  Test  Center  (ATC)  manages  YPG.  The  test  site  layout  was  prepared  to 
test  multiple  sensor  platforms  over  several  years.  The  calibration  and  blind  areas  had  all 
vegetation  removed.  For  Camp  Beale  the  site  preparation  was  organized  by  the  ESTCP  Office. 
Transect  surveys  were  organized  to  assess  target  density,  demonstration  grids  were  chosen  in 
collaboration  with  demonstrators,  and  targets  were  seeded  to  increase  the  number  of  buried 
UXO. 


6.3  SYSTEM  SPECIFICATION 

At  both  sites  the  MPV  was  programmed  with  two  different  acquisition  modes  for  detection  and 
cued  interrogation.  Both  modes  were  based  on  a  50%  duty  cycle  for  the  transmitter  so  that  the 
time  for  excitation  and  target  response  recording  were  equal.  At  YPG  the  highest  quality  data 
were  collected  during  cued  interrogation  for  classification  with  8  ms  excitation  and  time  decay, 
similar  to  typical  MetalMapper  parameters — the  MetalMapper  uses  similar  technology. 
Detection  was  set  to  a  relatively  long  2.6  ms  excitation  to  test  the  possibility  to  invert  the 
detection  data  and  perform  some  level  of  classification.  Forty-five  logarithmically  spaced  time 
gates  were  recorded  for  each  of  the  15  MPV  receiver  data  channels,  the  transmitter,  and  the  six 
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beacon  receivers.  After  the  YPG  demonstration  we  postulated  that  late-time  information  could 
improve  separation  between  targets  of  similar  size.  Therefore  we  used  a  longer  25  ms  excitation- 
decay  at  Beale  with  133  time  gates. 

Local  positioning  was  based  on  the  GPS  for  detection  survey  and  on  the  beacon  system  for  cued 
interrogation  (with  GPS  QC  in  open  field).  An  Applied  Physics  543  orientation  sensor  was 
mounted  at  the  far  end  of  the  MPV  handle  to  measure  azimuth.  The  three-axis  sensor  data  were 
also  used  for  verifying  the  pitch  and  roll  inferred  from  the  beacon  measurements.  The  beacon 
boom  was  laid  on  the  ground  within  2  m  of  the  interrogated  anomaly.  Boom  orientation  was 
recorded  by  placing  the  MPV  head  at  the  boom  center  and  lining  up  the  MPV’s  main  direction 
(Y-axis)  with  the  boom’s  direction.  To  facilitate  quality  checks,  the  boom  was  generally  oriented 
in  the  north-south  direction,  and  when  the  terrain  was  sloped,  the  boom  was  placed  uphill  from 
the  flag.  After  data  processing,  beacon-derived  positions  were  located  relative  to  the  local  flag 
and  the  geographic  North,  and  subsequently  globally-referenced  using  the  supplied  GPS 
coordinates  of  each  flag.  A  Novatel  RTK  GPS  was  also  mounted  on  the  MPV  handle.  It  was 
used  to  locate  preprogrammed  flag  locations  and  verify  beacon  accuracy  whenever  enough 
satellites  were  visible,  in  particular  at  the  IVS  and  in  the  open-field  “combined  area,”  where  we 
started  the  demonstration.  The  GPS  operated  at  20  Hertz;  its  base  station  was  located  at  a  marked 
monument,  whose  position  was  convenient  to  service  the  entire  site. 

6.4  DATA  COLLECTION 

6.4.1  Scale 

At  YPG,  the  Calibrations  Lanes  had  approximately  100  anomalies  distributed  over  0.3  acres  with 
2  m  spacing.  The  Blind  Grid  had  400  1  m  H  1  m  cells  with  a  potential  target  over  0.5  acres.  Both 
areas  were  surveyed  in  detection  and  cued  interrogation  mode. 

Beale  had  912  targets  distributed  in  four  different  areas  over  4  acres.  The  combined  area  had  259 
targets  distributed  over  1  acre  in  sloped,  open  field  to  allow  joint  testing  of  MetalMapper 
surveys.  The  nearby  area  had  250  targets  over  1  acre  of  terrain  with  dense  forest  and  steeper 
slope.  These  areas  had  multiple  cases  of  close  targets.  The  remaining  targets  were  buried  in 
sparsely  treed,  rolling  terrain. 

6.4.2  Survey  Pattern  and  Sample  Density 

At  YPG,  detection  had  the  field  operator  walk  along  preprogrammed  straight  lines  while 
sweeping  the  sensor  head  following  an  “S”  pattern  along  track  with  amplitude  and  period  of  0.5- 
0.6  m  for  full  coverage  of  1  m  wide  lanes.  Given  10  Hertz  update  rates  and  0.08  m  receiver  coils, 
sweeping  speed  was  kept  below  0.5  meters  per  second  (m/s),  which  corresponds  to  along-line 
speed  of  0.2-0. 3  m/s. 

Cued  interrogation  soundings  were  collected  around  the  expected  target  location.  At  YPG,  that 
location  was  determined  by  the  detection  survey  and  generally  coincided  with  the  center  of  a 
cell,  whereas  at  Beale  the  location  was  marked  by  a  flag  and  paint  on  the  ground.  Simulations 
have  suggested  that  a  five-point  pattern  made  of  the  comers  and  center  of  a  0.8  to  1  m  square 
centered  on  an  anomaly  is  sufficient  to  characterize  a  buried  target  (approximately  0.2  m  cube 
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spacing).  This  approach  was  successfully  applied  at  YPG.  At  Beale  we  found  at  an  early  stage 
that  there  could  be  significant  offsets  between  marked  locations  and  peak  anomaly.  Given  that 
there  could  also  be  multiple  targets,  we  adopted  a  conservative  approach  by  collecting  a  3H3 
point  grid  pattern  centered  on  the  flag  with  0.5-0. 6  m  station  spacing  (Figure  5).  An  additional 
sounding  was  collected  by  placing  the  MPV  sensor  head  above  the  anomaly  peak  at  45  E  pitch  to 
stimulate  a  different  excitation  direction  with  high  signal  while  remaining  as  close  as  possible  to 
the  target.  Only  one  tilt  test  was  performed  because  it  was  difficult  to  keep  the  sensor  steady  on 
its  side. 
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Figure  5.  Survey  pattern  and  tilt  test  for  cued  interrogation  with  MPV. 

Left:  Nine-point  survey  pattern  is  centered  on  target. 

Right:  Sensor  head  is  tilted  with  45  E  pitch. 


6.4.3  Quality  Checks 

System  checks  were  performed  by  surveying  the  IVS  on  a  daily  basis  at  Camp  Beale,  while  each 
survey  lane  started  and  finished  with  the  same  kind  of  calibration  target  at  YPG.  In-air 
measurements  were  acquired  several  times  a  day  to  monitor  sensor  drift  relative  to  battery 
power.  Geologic  background  had  a  weak  response  at  YPG,  thus  requiring  occasional  sampling 
only.  At  Camp  Beale  the  surface  material  had  the  distinct  red  tint  of  magnetite-rich  soil. 
Significant  soil  response  through  viscous  remanent  magnetization  (VRM)  was  detected  due  its 
typical  log-liner  time  decay  effect  on  vertical  and  radial  receivers.  One  background  sounding  was 
acquired  near  each  interrogated  anomaly. 

Field  soundings  were  reviewed  on  the  sensor  control  screen  immediately  after  acquisition. 
Recorded  decay  curves  were  displayed  to  locate  the  anomaly,  assess  spatial  coverage,  and  verify 
data  quality  (Figure  6).  Sufficient  coverage  was  controlled  by  verifying  that  the  receivers  furthest 
from  the  specified  target  location  had  weak  signal.  Otherwise,  additional  soundings  were 
collected.  Detected  abnormal  soundings  were  re-acquired  and  documented  in  field  notes.  If  a 
receiver  failed,  then  the  survey  was  interrupted  to  verify  that  the  failure  was  persistent  and 
consider  solutions.  David  George,  who  built  the  sensor,  participated  in  the  surveys  and  brought 
tools  and  spares  for  repair. 


19 


Figure  6.  Typical  target  response  above  a  buried  metallic  target. 

The  Z-component  data  show  that  the  target  is  closest  to  center  cube  (#3)  and  equally  distant  from 
lateral  cubes  2  and  4,  while  signal  in  5  seems  to  be  due  to  background.  The  Y  data  confirm  that 
target  is  buried  between  front  and  back  cubes  (1,5)  and  X  data  confirm  that  target  is  located 

between  side  cubes  2  and  4. 


Data  quality  was  also  reviewed  after  data  collection  (during  the  demonstration).  Any  issue  was 
investigated  and  potentially  problematic  anomalies  were  resurveyed.  For  instance,  we  found  at 
Beale  that  10  of  912  anomalies  had  insufficient  spatial  coverage.  This  could  be  due  to  large 
target  offset,  a  shrunk  survey  imprint  (operator  distraction),  or  absence  of  a  target  at  a  flag. 

Beacon-positioning  followed  a  specific  survey  protocol.  Beacon  signals  were  monitored  on  the 
sensor  display  similar  to  Figure  6  to  detect  perturbations  or  weak  signal.  Data  were  preprocessed 
and  visualized  onsite,  when  possible,  to  infer  beacon  locations,  compare  with  GPS,  and  verify 
accuracy. 

6.4.4  Data  Handling 

Data  were  stored  as  .tern  files  on  the  DAQ  and  converted  to  .csv  files  before  every  battery 
change.  A  copy  of  all  .tern  and  .csv  files  was  kept  on  the  DAQ  and  on  portable  hard-disk  drives. 
Field  notes  recorded  the  association  of  target  identities  and  file  numbers,  in  addition  to  any 
relevant  comment  for  nonstandard  events.  Notes  were  digitized  every  day  by  taking  pictures  of 
the  notes  and  filling  out  a  spreadsheet  to  be  used  for  preprocessing. 

6.4.5  Data  Summary 

Raw  data  sets  were  archived  by  both  SKY  and  G&G  Sciences.  Target  data  comprise  of  a 
configuration  file  and  an  electromagnetic  data  file.  Files  were  later  merged  and  preprocessed. 

6.5  VALIDATION 

Targets  were  seeded  at  YPG.  Validation  at  Beale  was  organized  by  the  ESTCP  Office. 
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7.0  DATA  ANALYSIS  AND  PRODUCTS 

7.1  PREPROCESSING 

7.1.1  Field  Data 

The  MPV  DAQ  computer  records  data  streams  from  the  sensor  head,  beacon  receivers,  attitude 
sensor,  and  GPS  with  the  GPS  time  stamp.  The  DAQ  saves  the  data  into  a  .tern  binary  file.  Each 
sounding  data  is  converted  to  a  .csv  file  without  any  data  alteration.  Several  preprocessing  stages 
are  perfonned  before  delivery  to  the  analysts. 

7.1.2  Positioning  Data 

The  beacon  receiver  data,  the  MPV  transmitter  current,  and  the  digital  compass  reading  are 
combined  to  infer  the  MPV  head  location.  The  method  is  similar  to  a  dipole  inversion  (Lhomine 
et  ah,  2011)  and  generally  yields  better  than  2  cm  accuracy.  When  the  GPS  has  maximum 
accuracy,  its  reading  and  the  attitude  sensor  data  can  be  utilized  to  predict  the  sensor  head 
location  for  comparison  with  the  beacon  prediction.  In  case  of  aberration  with  a  beacon  location, 
GPS  readings  and  field  notes  can  provide  information  to  diagnose  the  issue.  Otherwise  the 
sounding  may  be  discarded.  In  practice  there  was  no  detected  issue  with  the  beacon  at  Beale. 

7.1.3  Preprocessed  Data 

The  preprocessed  data  were  consolidated  in  individual  files,  each  containing  all  the  data 
associated  with  one  target:  EMI  data,  beacon-inferred  sensor-head  location,  GPS-derived  head 
location,  and  nearby  background  sounding.  The  EMI  data  were  nonnalized  by  the  transmitter 
current  to  remove  EMI  data  dependency  on  battery  power,  which  can  vary  by  up  to  20%.  In-air 
background  was  subtracted  from  the  EMI  data  to  remove  the  transmitter  ringing  that  can  still  be 
detected  at  early  time  (0. 1-0.3  ms).  The  data  were  reduced  to  25  (YPG)  or  32  (Beale)  time  gates 
by  logarithmic  integration  (through  the  same  process  that  had  reduced  the  4  microsecond  signal 
to  the  recorded  45  to  133  time  gates). 

At  Beale,  a  nearby  background  soil  signal,  clean  of  any  metallic  clutter,  was  associated  to  each 
target  for  the  analyst  to  devise  his  own  soil  correction.  One  complicating  factor  was  the 
occurrence  of  close  targets.  There  were  many  targets  with  less  than  1  m  spacing,  sometimes 
within  30  cm.  Under  such  circumstances  the  soundings  collected  between  the  two  neighboring 
targets  could  combine  responses  from  both  targets.  The  preprocessed  data  included  the  data  tiles 
for  both  targets  to  be  jointly  analyzed. 

7.2  TARGET  SELECTION  FOR  DETECTION 

At  YPG  target  detection  was  perfonned  in  real  time  based  on  the  MPV  detection  window 
(dancing  arrows,  signal  amplitude  and  time  decay)  and  verified  after  completion  of  data 
collection  by  generating  a  digital  map  of  the  recorded  dynamic  detection  data.  The  calibration 
lanes  were  surveyed  with  detection  thresholds  based  on  shakedown  tests.  The  thresholds  set  the 
sensitivity  of  the  arrows  display  to  obtain  high  sensitivity  to  low  amplitude  signal  using  linear 
scaling,  and  a  logarithmic  scaling  for  larger  amplitude  to  avoid  saturating  the  window  with  large 
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responses.  The  detection  rule  for  the  remainder  of  the  survey  (blind  grid  and  open  field)  was 
defined  after  evaluation  of  the  calibration  lanes,  where  targets  were  located  and  identified.  There 
was  a  detection  survey  in  this  study  at  Beale. 

7.3  PARAMETER  ESTIMATION 

The  preprocessed  data  were  imported  in  the  UXO  Lab  software  that  was  jointly  developed  by 
SKY  and  the  University  of  British  Columbia  and  used  in  numerous  ESTCP  and  SERDP  projects. 
Each  anomaly  was  treated  separately  and  generally  included  five- 10  soundings.  Data  were 
inverted  using  a  three-dipole  instantaneous  polarization  model.  In  the  EMI  method,  a  time 
varying  field  illuminates  a  buried,  conductive  target.  This  illumination,  or  primary  field,  induces 
currents  in  a  target  that  subsequently  decay,  generating  a  decaying  secondary  field  that  is 
measured  at  by  the  receivers  on  the  instrument.  In  the  UXO  community,  it  is  commonly  assumed 
that  the  secondary  field  can  be  accurately  approximated  as  a  point  dipole  (e.g.,  Bell  et  ah,  2001; 
Pasion  and  Oldenburg,  2001;  Gasperikova  et  ah,  2009).  The  process  of  estimating  the  target 
parameters  of  the  dipole  forward  model  from  the  data  is  called  data  inversion.  The  objective  of 
data  inversion  is  to  determine  the  parameters  that  most  accurately  predict  the  observed  data. 

The  target  parameters  include  the  position,  orientation,  and  polarizability  tensor  elements.  The 
magnitude  and  decay  of  the  polarizabilities  is  a  function  of  the  size  and  electromagnetic 
properties  of  the  target.  Like  principal  axes  of  an  ellipsoid,  the  magnitude  of  the  principal 
polarizabilities  (LI,  L2,  and  L3)  is  indicative  of  the  target  shape.  If  ordering  tensor  elements  such 
that  L1>L2>L3,  then  a  steel  body-of-revolution  (BOR)  would  have  L2=L3  for  a  rod-like  object 
and  L1=L2  for  a  plate-like  object. 

Inversion  setup  parameters  such  as  noise  estimation  were  decided  upon  examination  of  the 
training  data  and  background  measurements.  The  main  factors  affecting  data  quality  are 
instrument  noise,  environmental  noise,  and  positional  error.  The  first  two  mostly  depend  on  the 
sensor  characteristics  and  on  electromagnetic  interference  with  the  background  host  (e.g., 
magnetic  soil  at  Beale).  Baseline  background  was  subtracted  from  sensor  data  before  inversion; 
its  variability  was  utilized  to  estimate  expected  data  misfit  errors.  Positional  accuracy  is 
considered  as  data  quality  factor;  positional  error  is  equivalent  to  measurement  error  in  target 
response.  Therefore  data  errors  were  parameterized  as  a  floor  value  plus  a  percentage  of  the  data 
amplitude. 

Solutions  with  one  or  multiple  targets  were  always  tested.  At  highly  contaminated  sites, 
anomalies  often  overlap  due  to  the  close  proximity  of  multiple  targets.  The  inversion  strategy  for 
overlapping  targets  was  to  decompose  the  inverse  problem  into  several  steps.  Each  step  resolved 
a  subset  of  model  parameters.  The  first  step  was  to  solve  for  nonlinear  location  parameters  and 
subsequently  solve  for  linear  polarization  parameters.  With  an  optimal  estimate  of  locations  and 
dipolar  polarizations,  the  orientations  and  polarizability  decays  of  each  object  could  be  extracted 
from  estimated  magnetic  polarizability  tensors.  This  approach  was  successfully  tested  in 
numerous  ESTCP  studies.  Decisions  regarding  the  number  of  targets  at  a  given  location  were 
automatically  decided  through  statistical  classification  by  prioritizing  munitions-like  models. 
Inversion  results  were  reviewed  by  an  experienced  geophysicist  to  identify  any  potential  issues 
with  the  inversion  setup  or  the  sensor  data. 
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7.4  TRAINING  DATA 


The  classification  approach  was  based  on  the  capability  to  recognize  that  a  given  target  had 
similar  features  as  a  known  object.  The  method  therefore  required  a  library  of  known  target 
features  to  train  a  classifier.  The  classification  features  were  the  polarizability  decays,  which  are 
known  to  depend  on  target  size,  shape,  and  material  properties. 

At  YPG  the  target  library  was  built  on  calibration  lanes  data,  which  contained  14  different  types 
of  munitions  as  well  as  clutter.  These  were  buried  at  various  depths  and  orientations  and 
provided  an  opportunity  to  estimate  the  variance  of  these  features.  The  YPG  library  was  also 
utilized  at  Beale,  though  its  decay  range  of  8  ms  was  shorter  than  the  25  ms  decays  retrieved  at 
Beale.  Training  pit  and  IVS  data  completed  the  library.  Site-specific  training  data  were  also 
requested  for  distinctive  targets.  These  were  useful  to  characterize  unknown  UXO-like  objects, 
understand  complex  target  signatures  and  diminish  the  risk  of  misclassifying  unexpected  targets. 
A  clustering  algorithm  was  applied  to  identify  re-occurring  targets  that  could  require  particular 
attention.  The  method  was  based  on  comparing  all  recovered  polarizabilities  with  one  another 
and  grouping  self-similar  objects. 

7.5  CLASSIFICATION 

The  classification  approach  was  chosen  to  address  the  particular  classification  question  at  each 
site.  The  following  guiding  principles  were  adopted: 

•  Selection  of  models  in  classification:  We  assumed  that  classification  could  be 
perfonned  in  an  automatic  and  repeatable  manner.  Therefore  user  input  was 
limited  to  verifying  that  acceptable  fit  to  MPV  data  was  achieved.  Every  model 
meeting  that  criterion  was  considered  for  classification  without  further  user- 
dependent  pre-screening. 

•  Selection  of  features:  By  analysis  of  the  training  data,  those  features  that 
contribute  to  separation  of  the  different  classes  (comprising  UXO  types  and 
clutter)  were  selected. 

•  Choice  of  classification  algorithm:  Through  analysis  of  the  training  data,  the  best 
perfonning  classifier  was  selected.  We  focused  on  Support  Vector  Machine 
(SVM),  which  historically  had  yielded  the  best  results. 

•  Classification:  Anomalies  were  placed  in  a  prioritized  dig-list  by  using  the 
classifier  to  compute  probabilities  of  class  membership  for  unlabeled  feature 
vectors. 

•  Number  of  UXO-classes:  One  or  more  UXO  classes  could  be  defined  when 
seeking  UXO  of  widely  different  sizes.  When  multiple  UXO  classes  need  to  be 
used  (assume  there  are  M  of  them)  then  one  can  either  train  an  (M+l)-class 
classifier,  or  train  M  two-class  classifiers  and  combine  the  results. 

Historically,  classification  was  generally  based  on  a  combination  of  a  size  and  a  time  decay 
feature  for  the  early  ESTCP  classification  studies.  The  ESTCP  Camp  Butner  study  (MR-201004) 
showed  that  new-generation  sensors  such  as  the  Geometries  MetalMapper  could  support  richer 
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feature  vectors  for  classification  owing  to  their  extended  time  range  and  improved  recovery  of  all 
three  polarizability-component  decays.  Classification  was  improved  by  using  the  polarizability- 
decay  description  of  target  properties.  These  features  could  either  be  compared  to  library  items — 
through  a  misfit  metric — or  be  used  with  a  statistic  classifier  such  as  SVM,  which  can 
accommodate  nonlinearity  and  large  feature  vectors  and  can  include  multiple  targets  within  a 
given  class. 

The  SVM  framework  was  preferred  for  both  studies.  At  YPG  the  main  reason  was  the  large 
variety  of  munitions  types:  comparing  library  misfits  and  defining  thresholds  for  14  munitions 
types  seemed  more  hazardous  than  grouping  similar  size  targets  in  four  to  six  classes.  At  Beale 
the  presence  of  magnetic  soil  could  produce  secondary  models  accounting  for  the  soil.  These 
models  could  have  similar  amplitude  and  decay  as  an  81  mm  or  105  mm  projectile  and  thus 
could  bias  a  library  misfit,  whereas  SVM  could  be  systematically  trained  to  dismiss  soil  models 
without  any  manual  intervention  (a  trained  geophysicist  could  eliminate  these  models  during 
inversion  QC).  The  SVM  classifier  was  trained  on  each  class  to  separate  that  class  against  the 
other  classes  and  clutter.  The  SVM  classifiers  were  then  applied  to  all  field  targets.  For  each 
class  the  SVM  statistic  indicated  the  likelihood  that  a  target  belonged  to  that  class.  Targets  could 
be  given  priority  for  digging  by  sorting  the  SVM  statistic  by  decreasing  order. 

The  ranked  classification  anomaly  list  was  fonnatted  as  shown  in  Figure  7.  The  first  items  on 
each  anomaly  list  were  the  requested  training  targets.  Next  were  to  be  those  targets  for  which 
reliable  parameters  cannot  be  extracted  and  therefore  had  to  be  dug.  In  practice  data  quality  was 
such  that  we  considered  that  every  anomaly  could  be  analyzed.  The  list  continued  with  the  “high 
confidence  munitions”  targets.  Items  were  ranked  according  to  decreasing  confidence  that  the 
item  was  hazardous.  Any  items  that  were  analyzed  without  reaching  an  unambiguous 
classification  decision  were  placed  next  on  the  anomaly  list.  Finally,  all  items  that  were 
confidently  classified  as  nonhazardous  were  ranked  by  their  confidence. 


Rank 

Anomaly 

ID 

Decision 

Statistic 

Category 

Dig  on 
First 

Pass 

Comment 

-9999 

2498 

-9999 

0 

1 

Can’t  extract  reliable  features 

N 


.03 


.03 


.02 


.01 


High  confidence  non-munition 


Figure  7.  Format  of  prioritized  anomaly  list  to  be  submitted  to  the  ESTCP  Office. 
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7.6  DATA  PRODUCTS 

The  primary  data  products  were  the  prioritized  dig  lists  as  described  above  for  each  site. 
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8.0  PERFORMANCE  ASSESSMENT 


8.1  DETECTION  OF  MUNITIONS  OF  INTEREST 

The  MPV  detection  survey  method  tested  at  YPG  was  simply  based  on  detecting  metallic 
objects.  Having  smaller  receiver  coils  and  keeping  closer  to  the  ground  than  an  EM61  cart 
survey,  the  MPV  can  potentially  detect  and  locate  much  metallic  clutter  such  as  nails  and  bolts. 
Therefore  the  MPV  detected  equally  well  munitions  and  clutter  at  YPG  with  over  90%  detection 
of  munitions  and  over  90%  false  positives,  though  there  was  false  alarm  from  background.  The 
relatively  high  sensitivity  to  small,  shallow  clutter  could  be  compensated  through  data-based 
logic,  such  as  detection  on  few  receivers  and  fast  time  decay.  More  detection  data  will  be 
collected  in  upcoming  demonstrations  to  support  clutter-rejection  analyses. 

8.2  CORRECT  CLASSIFICATION  AND  REDUCTION  OF  FALSE  ALARMS 
8.2.1  Stability  of  the  Recovered  Target  Parameters 

Classification  relied  on  comparison  with  reference  target  signatures — namely  their  polarizability 
decay  curves.  The  inferred  polarizabilities  for  each  munitions  types  are  shown  in  Figure  8  for 
YPG  Calibration  Lanes  (Blind  Grid  identity  are  not  available  to  preserve  the  test  site)  and  in 
Figure  9  for  field  targets  at  Beale.  For  both  studies  the  tight  distributions  of  polarizability  decay 
curves  indicate  that  data  quality  was  sufficiently  high  to  recover  at  least  two  of  three 
polarizabilities.  Predicted  models  were  stable  and  could  support  relatively  aggressive 
classification  using  2-3  polarizability  decays. 


Figure  8.  Recovered  polarizabilities  of  training  targets  at  YPG  calibration  lanes  (8  ms  decay). 

Main  polarizability  (LI)  is  showed  in  red;  secondary  polarizabilities  (L2,  L3)  are  showed  in  blue  and  green.  Results 
are  shown  for  single-target  inversions,  hence  the  parameter  variance  for  “60  mm  and  clutter.” 
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Figure  9.  Recovered  polarizabilities  for  Camp  Beale  field  targets  (25  ms  decay). 

Note  several  fuze  types.  Large  fuzes  were  recognized,  though  no  fuze  was  deemed  of  interest. 

8.2.2  Prioritized  Dig  List 

At  YPG,  the  rate  of  correctly  classifying  munitions  reached  95%  for  shallow  depth  and  90%  for 
medium  depth  (Table  5  of  ATC  report).  Detection  and  classification  rates  were  similar;  therefore 
most  detected  munitions  were  properly  identified.  Caliber  identification  was  88%  for  small 
munitions  and  57%  for  medium  and  large  ones  (Table  7  of  ATC  report).  The  lower  rates  are 
explained  by  the  SVM  method,  in  which  munitions  were  grouped  in  size  classes  instead  of 
seeking  to  recognize  each  individual  target  type. 

For  Beale  classification  performance  is  illustrated  in  Figure  10  in  the  form  of  receiver  operating 
characteristic  (ROC)  curves.  Note  that  the  presence  of  fuzes  disrupted  the  classification  process, 
as  large  fuzes  were  originally  considered  as  TOI  but  scored  otherwise.  All  89  TOI  were 
recovered  after  approximately  95  pieces  of  clutter  had  to  be  excavated,  leaving  over  700  items  in 
the  ground — 88%  clutter  rejection  rate.  Efficiency  could  have  been  even  higher  in  the  absence  of 
fuzes.  Having  retrained  and  applied  the  classifier  in  a  fully  automated  manner  without  fuzes,  we 
found  that  only  48  pieces  of  clutter  would  need  to  be  excavated  to  retrieve  all  TOI  (Figure  10B). 
The  presence  of  fuzes  also  promoted  a  conservative  stop-digging  point  because  fuzes  tend  to 
have  low  SNR  and  less  reliable  polarizabilities. 
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A:  Submitted  ROC  trained  to  search  for  large  fuzes.  “Can’t  decide”  category  (yellow)  corresponds  to 
targets  that  were  predicted  to  be  more  than  0.5  m  away  from  their  associated  flag. 

B:  ROC  curve  without  fuzes. 


8.2.3  Data  Quality 

Success  at  correctly  identifying  TOI  and  keeping  low  false  alann  rate  (FAR)  suggests  that 
relatively  high  quality  data  were  obtained  with  the  MPV.  The  expectation  for  a  handheld  sensor 
is  the  detection  and  classification  of  shallow  targets — larger  systems  can  investigate  deeper 
targets  through  use  of  larger  transmitters  and  higher  power.  The  YPG  demonstration  showed  that 
100%  of  munitions  were  detected  and  95%  were  correctly  classified  within  0.3  m  depth.  These 
probabilities  remained  high  at  90%  for  each  stage  within  1  m  depth.  As  expected,  and  by  design, 
performance  is  much  poorer  at  greater  depth.  All  munitions  were  buried  within  0.45  m  at  Camp 
Beale  and  100%  correct  classification  was  achieved. 

8.3  LOCATION  AND  DEPTH  ACCURACY 

This  section  addresses  accuracy  of  the  beacon  positioning  technology  for  locating  the  MPV  and 
accuracy  of  the  predicted  target  location.  The  former  was  critical  to  the  Beale  demonstration  as 
GPS  could  not  be  used  as  an  alternative.  Beacon  accuracy  was  tested  by  comparing  predicted 
positions  with  GPS  and  beacon  during  cued  interrogation  both  at  YPG,  where  open  field 
conditions  prevailed,  and  at  Beale,  where  the  IVS  and  combined  area  had  open  sky  view.  Given 
the  accuracy  of  the  RTK  GPS  and  attitude  sensor  and  their  location  relative  to  the  MPV  sensor 
head,  the  head  location  can  be  predicted  with  approximately  2  cm  accuracy.  Results  presented  in 
Figure  1 1  and  in  Lhomme  et  al.  (2011)  show  that  similar  accuracy  was  achieved  with  the  beacon, 
which  directly  predicts  the  MPV  transmitter  location  and  orientation — azimuth  is  irresolvable 
and  obtained  with  the  compass  heading.  In  the  YPG  technical  report,  we  showed  that  the 
addition  of  2  cm  positional  and  2E  attitude  errors  had  minimal  effect  on  the  recovered 
polarizabilities. 
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Figure  11.  Relative  accuracy  of  beacon  positioning  system  compared  with  GPS. 

The  beacon  method  predicts  MPV  transmitter  location.  The  GPS  and  attitude  sensor  are  mounted  at  the 
far  end  of  the  MPV  handling  boom  and  can  also  be  used  to  predict  the  MPV  transmitter/receiver  head 
location  with  2  cm  accuracy  in  open  field  (1.5  m  lever  arm). 

Relative  positional  error  was  compared  in  open  field. 

Ability  to  locate  an  anomaly  source  is  critical  for  reliably  retrieving  target  parameters  and 
perfonning  classification.  The  YPG  Calibration  Lanes  provide  ideal  conditions  for  verifying 
location  and  depth  accuracy  since  targets  were  seeded  at  the  center  of  each  cell.  Location  and 
depth  errors  were  normally  distributed  with  less  than  0.01  m  mean  and  0.07  m  standard 
deviation.  Targets  were  well-located  due  the  information  from  horizontal  receiver  components 
that  sense  the  crossover  in  scattered  field  direction  above  the  anomaly  source.  At  Beale, 
geographic  location  was  generally  correct  within  0.2  m,  which  was  satisfactory  given  survey 
uncertainty  under  tree  canopy.  Depth  was  matched  within  0.05  m  for  most  TOI,  with  expected 
larger  spread  for  81  mm  and  105  mm,  as  their  slow  decay  rates  resembled  that  of  magnetic  soil. 

8.4  PRODUCTIVITY 

Ultimately,  the  time  and  cost  associated  with  cued  interrogation  and  classification  of  all  detected 
anomalies  and  excavation  of  selected  anomalies  should  be  smaller  than  that  of  a  UXO-technician 
excavating  every  anomaly  with  a  shovel.  Survey  time  is  determined  by  the  number  of  cued 
interrogation  soundings  and  any  additional  time-consuming  field  procedure.  Classification  time 
depends  on  the  amount  of  non- automated  user  input  that  is  required  for  data  analysis. 

8.4.1  Data  Collection 

The  original  MPV  procedure  for  cued  interrogation  was  based  on  adaptive  sampling  of  a  field 
anomaly  with  the  assumption  that  five-six  soundings  would  be  sufficient  if  the  anomaly  was  well 
located.  That  approach  was  successfully  applied  at  YPG,  where  single  targets  were  seeded  at 
known  locations.  The  survey  was  not  truly  adaptive  though,  since  no  fonnal  metric  was  derived 
after  each  sounding  acquisition.  Instead,  the  operator  was  instructed  to  follow  a  virtual  square 
pattern  and  add  soundings  if  the  signal  was  found  to  extend  beyond  that  pattern.  Subsequent  data 
analysis  showed  that  as  few  as  three  high-fidelity  soundings  could  be  sufficient  for  classification 
of  many  targets.  Live-site  conditions  have  positional  offsets  between  flags  and  sensed  anomalies 
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and  potential  occurrence  of  multiple  targets.  At  Beale,  the  standard  operating  procedure  (SOP) 
was  modified  to  a  conservative  9-point  grid,  which  decreased  productivity  while  alleviating 
reliance  on  operator.  In  retrospect  we  re-inverted  the  data  after  selecting  only  the  5 -point  square 
pattern  and  found  that  similar  classification  perfonnance  could  be  obtained.  These  results 
suggest  that  fewer  soundings  could  be  collected  in  future  deployments  while  keeping  similar 
classification  perfonnance.  The  average  survey  rate  at  Beale  was  approximately  4  minutes  per 
target  and  90  targets  per  day.  That  included  moving  the  equipment,  positioning  the  beacon  boom 
and  acquiring  one  sounding  on  top  of  it  to  register  azimuth  (30  seconds),  collecting  10  soundings 
with  6.3-second  stacks,  waiting  2-10  seconds  for  the  sensor  display  to  register  each  sounding, 
and  collecting  and  verifying  a  background  sounding  (15-20  seconds).  Productivity  could  be 
improved.  Reduction  from  10  to  5  soundings  per  target  would  save  approximately  1  minute,  thus 
increasing  productivity  by  30%.  Depending  on  environmental  conditions,  in  particular  the 
magnetic  soil  setting,  fewer  background  soundings  could  be  collected. 

Productivity  could  improve  through  hardware  upgrades.  A  proper  touch-screen  monitor  wired  to 
the  DAQ  would  reduce  lags,  instead  of  the  current  setup,  which  relies  on  the  monitor  of  a 
Panasonic  Toughbook  computer  or  an  Apple  iPad  linked  to  the  DAQ  through  marginally  stable 
wi-fi  network  with  limited  bandwidth  remote  desktop  protocol  (RDP).  An  audible  notification 
after  stacks  are  completed  would  further  speed  acquisition  by  telling  the  operator  to  move  on. 
Beacon  azimuth  registration  is  time  consuming  because  the  MPV  head  has  to  be  lined  up  with 
the  beacon  boom  with  great  care.  At  least  20  seconds  could  be  saved  and  positional  accuracy 
could  be  improved  by  installing  a  dedicated  compass  on  the  beacon  boom.  A  standard  digital 
compass  with  1 E  accuracy  would  be  more  reliable  than  a  careful  operator.  This  would  require 
minor  modifications  to  the  data  format  to  accommodate  this  new  data  stream. 

8.4.2  Data  Management  and  Analysis  Time 

Procedures  were  developed  and  tested  as  part  of  these  demonstrations  to  meet  the  challenges  of 
handling  a  new  sensor  and  managing  large  amounts  of  live-site  data.  Therefore  productivity  was 
lower  than  expectations  for  a  more  mature  system. 

Data  management  took  significant  time  (5  minutes  per  target).  With  over  900  targets  with  10 
soundings  per  target  and  two  files  per  sounding,  Beale  generated  approximately  20,000  files  for 
which  names,  and  sequence  numbers  had  to  be  matched  with  targets  through  spreadsheets.  This 
process  could  be  easier  with  a  strict  data  collection  such  that  file  number  sequence  would  be 
invariable  and  not  rely  on  manual  spreadsheet  entries.  Though  automated,  computation  of 
beacon-inferred  locations  required  the  analyst  to  visually  compare  with  GPS  predictions  until 
sufficient  confidence  in  the  process  was  gained.  The  process  is  now  stable  and  should  require 
less  manual  QC  in  the  future.  Background  effects  had  to  be  investigated  at  Beale  to  characterize 
their  amplitude,  variability,  and  spatial  distribution. 

At  the  data  interpretation  level,  methods  had  to  be  devised  and  tested  for  defeating  potential  soil 
disturbance  through  background  compensation  and  inversion  of  soil  sensitive  receiver  data. 
Close  targets  had  to  be  identified  and  jointly  processed  to  account  for  potential  interaction  and 
overlapping  signals.  The  task  was  occasionally  complicated  by  ambiguity  on  the  expected  target 
to  be  characterized.  These  cases  required  manual  intervention  to  mask  and  interpret  the  data  and 
apply  close  scrutiny  to  inversion  results.  Specific  classification  algorithms  were  implemented 
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with  multistage  cascading  SVM  to  seek  14  target  types  at  YPG.  The  framework  was  easily 
transferable  to  the  Beale  study,  though  special  methods  for  automatically  identifying  and 
weaning  off  soil  models  had  to  be  developed  and  incorporated.  We  expect  that  data  processing 
will  be  faster  for  the  upcoming  demonstrations  through  use  of  the  numerous  algorithms  and  QC 
protocols  that  were  implemented  as  part  of  this  project. 

8.5  FIELD  USABILITY 

Field  usability  is  critical  for  future  technology  adoption.  It  was  tested  by  having  the  principal 
investigator  (PI)  and  a  field  geophysicist  trained  to  set  up  surveys,  program  data  acquisition 
parameters,  and  collect  data  in  detection  and  cued  interrogation  mode. 

Programming  data  collection  parameters  through  G&G  EM3DAcquire  user  interface  was 
relatively  straightforward.  Parameters  from  previous  surveys  could  easily  be  adapted  and 
modified  to  serve  a  new  survey.  Survey  lines  and  cued  locations  had  to  be  prepared  before  the 
survey  (e.g.,  in  Oasis  Montaj  or  Matlab)  and  saved  in  a  specifically  formatted  text  file. 
Programming  of  the  GPS  base  station  was  a  complex  and  time  consuming  task  and  required  use 
of  external  communication  software  such  as  HyperTenninal.  As  a  result,  there  was  no  integrated 
method  for  troubleshooting  peripheral  sensors  such  as  GPS  and  the  attitude  sensor.  This  caused 
delays  in  setting  up  reference  coordinates  at  the  beginning  of  the  survey,  and  further  delays  when 
these  peripherals  were  suspected  to  cause  system  instability  and  stop  the  survey.  The  problem 
was  particularly  acute  at  YPG,  when  the  MPV’s  computer  had  trouble  keeping  stable 
communication  between  its  peripherals.  This  was  mostly  due  to  low  signals  and  bandwidth  via 
the  wireless  network — required  for  the  control  display — and  volatility  of  the  Windows  7 
operating  system.  At  YPG  the  MPV  computer  had  to  be  restarted  numerous  times  without  any 
troubleshooting  protocol.  The  problem  was  mostly  resolved  for  the  Beale  survey. 

Data  collection  in  detection  and  cued  interrogation  mode  were  quickly  assimilated  and  generally 
resulted  in  high  perfonnance.  In  detection  mode,  most  metallic  targets  were  found  and  there  was 
no  false  detection.  Survey  conditions  were  relatively  easy  on  the  YPG  Blind  Grid.  However, 
detection  was  significantly  more  difficult  in  the  Desert  Extreme  area,  where  the  operator  was 
tasked  to  monitor  detection  arrows  and  spatial  coverage  while  keeping  the  sensor  head  close  to 
the  ground  and  watching  for  vegetation  and  surface  relief.  This  process  would  benefit  from 
automated  alarms  for  target  detection  and  spatial  coverage  gaps.  High  quality  data  were 
generally  collected  in  cued  interrogation.  There  was  no  need  to  resurvey  any  anomaly  at  YPG, 
where  targets  were  always  located  at  the  center  of  a  cell.  At  Beale  the  crew  was  sent  to  resurvey 
1%  of  the  anomalies  (10  out  of  912  targets).  Targets  required  resurvey  because  their  peak 
anomaly  was  significantly  offset  from  the  flag  location,  resulting  in  insufficient  spatial  coverage. 
The  current  display  requires  constant  focus  for  the  operator  to  locate  detected  anomalies;  this 
taxing  effort  could  be  relieved  with  a  data  gridding  display  capability. 

In  tenns  of  hardware,  the  sensor  head  was  maneuverable  with  one  or  two  arms  and  was 
sufficiently  robust  not  to  incur  any  damage.  The  backpack  with  DAQ  and  batteries  felt  heavy 
(approximately  40  pounds),  especially  at  Beale  where  missing  power  cables  forced  use  of  a 
metal  enclosure  with  four  batteries — instead  of  no  case  and  two  batteries — thus  increasing 
weight  by  approximately  10  pounds.  Receiver  cable  connectors  with  the  DAQ  were  fragile:  two 
beacon  cables  were  pulled  and  damaged  at  YPG;  two  MPV  receiver  cable  connectors  were 
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damaged  at  Beale,  causing  significant  challenge  for  diagnostic  and  survey  delays  of  1.5  days. 
Although  spares  will  be  available  for  the  upcoming  deployments,  standard  troubleshooting 
procedures  have  yet  to  be  defined. 

Overall  the  technology  is  generally  reliable  and  has  a  strong  potential  for  successful  field  surveys 
and  classification  in  future  ESTCP  Live-Site  Demonstrations. 
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9.0  COST  ASSESSMENT 


The  figures  presented  hereafter  reflect  the  costs  incurred  to  date  for  system  maintenance, 
software  modifications  and  upgrades,  shakedown  testing,  and  deployment  and  analysis  of  data 
collected  during  the  Camp  Beale  demonstration.  The  costs  pertain  to  a  prototype  technology  and 
are  expectedly  higher  than  commercial  operation,  as  there  remained  development  tasks,  sensor 
data  characterization  studies,  and  instrument  issues  that  had  to  be  addressed.  Some  of  the  cost 
will  likely  decrease  as  the  technology  matures,  survey  procedures  get  fonnalized  and  data  QC 
tools  and  data  processing  methods  become  stable. 

9.1  COST  MODEL 

Time  and  resources  were  tracked  to  assess  the  cost  of  deploying  the  technology  for  future  live 
sites.  The  site  had  particular  environmental  conditions  (active  magnetic  soil)  that  required  in- 
depth  background  noise  analysis  and  testing  of  alternative  parameter  extraction  and  classification 
strategies  for  defeating  potential  noise  disturbance.  Identification  of  closely  spaced  targets  was 
also  a  new  challenge  for  MPV  surveying  and  required  special  care  at  the  parameter  extraction 
stage.  A  cost  model  is  proposed  in  Table  2  based  on  field  survey  with  three  people,  a  $104 
hourly  rate  for  all  personnel,  and  interrogation  of  912  anomalies  over  a  15-day  period.  Costs  of 
planning  field  deployment,  retrospective  analysis,  and  reporting  are  not  included. 

Table  2.  Cost  model  for  MPV  demonstration  at  former  Camp  Beale. 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated 

Hours 

Estimated  Cost 

Instrument  cost 

Component  costs  and  integration  costs 

•  Engineering  estimates  based  on  current 
development 

•  Lifetime  estimate 

•  Consumables  and  repairs 

N/A 

N/A 

5  days 

N/A 

$10,000 

Survey  cost 

400  hours 

$45,000 

Mobilization  and 
demobilization 

Cost  to  mobilize  to  and  demobilize  from  site 

•  Travel  to  town  base  (time+ticket) 

•  Daily  travel  to  site  (time+car  rental) 

10  h/ppl/trip 

2  hours 

Pre-survey  activities 

Before  mobilization:  Cost  of  one-time  event 

•  Sensor  verification  at  G&G  workshop 

On  site:  Cost  of  one-time  preparation  and  testing 

•  Programming  of  GPS 

•  Preparation  of  target  location  list 

•  Collection  of  training  data  (test  pit) 

Preparation  of  QC  tools:  Total  cost 

•  Hours  per  anomaly 

•  Personnel  required 

20  hours 

2  hours 

4  hours 

3x4  hours 

40  hours 

1  geophysicist 

$2000 

$200 

$500 

$1500 

$4000 

Instrument  setup 
costs 

Unit:  Set  up,  calibrate  at  IVS  and  review  data 

•  Hours  required 

•  Personnel  required 

•  Frequency  required 

0.3  hours/ppl 

2  ppl 

Daily 

$1500 
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Table  2.  Cost  model  for  MPV  demonstration  at  former  Camp  Beale  (continued). 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated 

Hours 

Estimated 

Cost 

Data  collection 

costs 

Unit:  Cost  of  collecting  data  per  anomaly 

•  Hours  per  anomaly 

•  Personnel  required 

4  min 

2  field  ppl 

$13,000 

Preprocessing  costs 

Unit:  Cost  per  anomaly 

•  Hours  per  anomaly 

•  Personnel  required 

4  min 

1  geophysicist 

$6500 

Discrimination  data  processing 

255  hours 

$30,000 

Data  extraction 

Unit:  Cost  per  anomaly 

•  Time  required 

•  Personnel  required 

3  min 

1  geophysicist 

$5000 

Parameter  extraction, 
including  inversion 
and  review 

Analysis  of  local  background  noise:  Total  cost 

•  Time  required 

•  Personnel  required 

Unit:  Cost  per  anomaly 

•  Time  required 

•  Personnel  required 

60  hours 

1  geophysicist 

3  min 

1  geophysicist 

$6500 

$5000 

Classifier  training 

Define  strategy  for  defeating  soil:  Total  cost 

•  Time  required 

•  Personnel  required 

Unit:  Cost  per  anomaly 

•  Time  required 

•  Personnel  required 

40  hours 

1  geophysicist 

1  min 

1  geophysicist 

$4000 

$2000 

Classification  and 
production  of 
prioritized  dig  lists 

Unit:  Cost  per  anomaly  (multiple  list  stages) 

•  Time  required 

•  Personnel  required 

3  min 

1  geophysicist 

$5000 

9.2  COST  DRIVERS 

The  MPV  was  developed  to  provide  a  moderate  cost,  reliable  portable  sensor  with  advanced 
discrimination  capabilities  that  can  operate  at  sites  with  challenging  surveying  conditions.  As  a 
portable  system,  deployment  logistics  and  costs  for  transport  and  operation  are  quite  low  relative 
to  towed  arrays  or  other  vehicular-based  systems.  The  primary  costs  are  incurred  for  labor  and 
travel  for  the  operators,  and  the  primary  cost  driver  becomes  the  duration  of  deployment,  directly 
related  to  the  acreage  to  be  surveyed  as  well  as  the  difficulty  of  the  terrain  (steep,  rocky,  very 
uneven,  and  wooded  terrain  will  take  somewhat  longer  to  survey  just  because  it  is  more  difficult 
to  hike  across  these  areas). 

9.3  COST  BENEFITS 

The  primary  driver  for  developing  the  MPV  is  to  make  discrimination  feasible  at  a  wide  range  of 
sites  where  field  conditions  prohibit  the  use  of  cart-based  systems,  and  for  small-scale 
deployment  where  a  small  area  needs  to  be  surveyed  or  where  anomalies  need  to  be  resurveyed 
at  a  lower  cost  than  a  cart -based  system. 
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10.0  IMPLEMENTATION  ISSUES 


Use  of  the  MPV  technology  is  not  expected  to  be  affected  by  any  particular  regulation.  The  MPV 
has  low  impact  on  its  environment  owing  to  its  handheld  deployment  mode.  Though  GPS 
positioning  uses  regulated  radio  frequency,  there  is  no  reliance  on  GPS  as  cued  interrogation  is 
supported  with  the  beacon  and  detection  surveys  can  afford  lower  spatial  accuracy  requirements, 
for  instance,  with  cotton  thread  and  spool. 

The  original  implementation  objectives  were  to  prove  that  high  quality  data  could  be  collected 
and  support  reliable  classification,  to  verify  that  such  perfonnance  could  be  achieved  within  a 
handheld  form  factor  that  could  sustain  field  use,  and  to  assess  viable  deployment  modes. 
Detection  and  classification  were  clearly  demonstrated.  Maneuverability  and  ruggedness  were 
considered  to  be  adequate  by  our  team,  though  commercial  users  have  not  yet  been  confronted 
with  the  technology.  The  MPV  still  has  fragile  hardware  parts  that  can  readily  be  protected. 
Different  survey  modes  were  tested  and  productivity  is  expected  to  improve  as  the  volume  of 
data  to  collect  can  be  reduced  without  compromising  perfonnance. 

The  MPV  tested  in  this  study  is  a  second-generation  prototype  built  by  G&G  Sciences.  It  is  not 
commercially  available  and  only  one  government-owned  unit  currently  exists,  and  its 
maintenance  has  to  be  sourced  to  G&G.  Basic  components  are  commercially  available — receiver 
cubes  and  National  Instrument  DAQ  computer  and  receiver  modules — while  receiver  filters, 
transmitter  coils  and  switches,  and  DAQ  software  (EM3DAcquire)  are  G&G  custom  builds  that 
cannot  be  replicated  or  maintained  by  other  vendors.  The  MPV  requires  an  attitude  and  heading 
sensor;  there  is  no  standard  communication  protocol  and  few  models  are  currently  supported, 
which  can  incur  significant  costs  if  alternative  attitude  sensors  are  to  be  integrated  such  that  they 
are  compatible  with  EM3DAcquire.  Any  GPS  can  be  used  as  long  as  it  is  set  to  send  a 
standardized  data  sequence  (e.g.,  NEMA  fonnats). 

The  MPV  is  relatively  straightforward  to  operate  in  the  field.  Field  crews  were  quickly  trained  to 
collect  data  and  define  basic  principles  for  locating  the  signal  origin  and  detecting 
malfunctioning  receivers.  Note  that  crews  had  significant  experience  with  geophysical  data 
analysis,  which  was  useful  for  immediate  review  and  interpretation  of  the  recorded  data.  Setting 
up  and  troubleshooting  communication  with  peripherals,  GPS,  and  attitude  sensor  requires 
experience  of  less  user-friendly  tools.  Experience  with  setting  up  GPS  is  useful.  Troubleshooting 
DAQ  software  crashes  is  currently  difficult. 

Instructions  on  survey  protocols,  such  as  acquisition  parameters  and  cued  survey  patterns,  are 
contained  in  the  YPG  and  Camp  Beale  reports.  General  guidance  on  EM3DAcquire  can  be  found 
on  the  Geometries  website  for  the  MetalMapper  sensor,  which  uses  similar  underlying 
technology.  At  this  point,  we  would  recommend  that  an  experienced  geophysicist  be  associated 
with  any  data  collection  to  guide  field  operators  and  frequently  verify  data  quality. 
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